Adaptive virtual keyboard

ABSTRACT

A method allows a portion of a virtual keyboard to be partially invisible and allows a user to activate keys in that portion when the invisible part is hit. According to an exemplary embodiment, the method is used in a touch screen device and includes steps of: presenting a virtual keyboard, wherein all of the virtual keyboard is displayed on the screen and falls on a first portion of touch-sensitive elements; moving the virtual keyboard, in response to a user input, to a position in which some keys including a first key of the virtual keyboard are partially displayed on the screen and are visible, the visible portions of those partially displayed keys fall on the first portion of the touch-sensitive elements and invisible portions fall on the second portion of the touch-sensitive elements, the first key covers a first portion of the first portion of the touch-sensitive elements and a first portion of the second portion of the touch-sensitive elements; and when the first key is hit, performing a particular function assigned to the first key.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to methods and apparatuses forpresenting virtual keyboards on a touch screen, and more particularly,to methods and apparatuses for presenting a virtual keyboard with somekeys partially or fully extend beyond the screen, dynamically adapting avirtual keyboard to a particular typing habits of a user, and allowing auser to customize a keyboard layout including adding/deleting keys andchanging key positions.

2. Background Information

A “virtual” or “soft” keyboard is a keyboard depicted on a touch screenand the keystrokes or hits are registered by a touch detecting function.It is not a hardware keyboard, which is located external to the touchscreen.

Many modern hand-held computers, such as a tablet, have a touch screenand usually provide a virtual keyboard on the touch screen. Often thesize and key arrangement of the virtual keyboard do not match the sizeof the writer's hands, for reasons such as the touch screen size isfixed and limited, and the sizes of palms and fingers are different fromwriter to writer. As touch screens become cheaper, touch screens capableof displaying a full size virtual keyboard similar to external standardkeyboards become more and more popular. However, the full size virtualkeyboard is still fixed in key arrangement, and fixed in size, shape,and relative distance between two keys. As such, it cannot match thehands of all writers. Accordingly, there is a need in the art to addressthe foregoing issues and thereby provide improved methods andapparatuses for presenting virtual keyboards in a touch screen device.

SUMMARY OF THE INVENTION

In accordance with an aspect of the present invention, a touch screendevice is disclosed. The touch screen device comprises a screen;touch-sensitive elements, a first portion of the touch-sensitiveelements being arranged inside the screen and a second portion of thetouch-sensitive elements is arranged outside of the screen; and aprocessor coupled to the screen and the touch-sensitive elements, theprocessor outputting a first virtual key to a first portion of thescreen and moving the first virtual key from the first portion of saidscreen responsive to a user input to a position in which to the firstvirtual key is partially displayed on the screen and covers a firstportion of the first portion of the touch-sensitive elements and aninvisible portion of the first virtual key covers a first portion of thesecond portion of the touch-sensitive elements, the first virtual key ifhit causing the processor to perform a particular function.

In one embodiment, the processor is operative to detect a signalindicating which one of the touch-sensitive elements has been hit andresponsive to detecting of the first portion of the first portion of thetouch-sensitive elements being hit, the processor executes theparticular function.

In another embodiment, responsive to detecting of the first portion ofthe second portion of the touch-sensitive elements being hit, saidprocessor may also execute the particular function.

In yet another embodiment, the processor moves the first virtual key toa position in which all of said first virtual key is invisible andcovers a second portion of the second portion of the touch-sensitiveelements and responsive to detecting of the second portion of the secondportion of the touch-sensitive elements (320) being hit, the processorexecutes said particular function.

In yet another embodiment, the processor outputs a virtual keyboardincluding the first virtual key and moves the virtual keyboard to aposition in which some virtual keys including the first virtual key arepartially displayed on the screen and fall on the first portion of thetouch-sensitive elements, and invisible portions of those virtual keysfall on the second portion of the touch-sensitive elements, responsiveto detecting of the first portion of the first portion of thetouch-sensitive elements being hit, the processor executes theparticular function, and responsive to detecting of the first portion ofthe second portion of the touch-sensitive elements being hit, theprocessor also executes the particular function.

In yet another embodiment, the processor moves the virtual keyboard to aposition in which some virtual keys are completely invisible and fall onsaid second portion of said touch-sensitive elements, the first virtualkey covers a second portion of the second portion of the touch-sensitiveelements, and responsive to detecting of the second portion of thesecond portion of the touch-sensitive elements being hit, the processorexecutes the particular function.

In one embodiment, the first and second portions of the touch-sensitiveelements are next to each other in a consecutive manner. In anotherembodiment, the first and second portions are non-consecutive. Forexample, the screen is located in a first side of the touch screendevice and the second portion is located on a second side other than thefirst side of the touch screen device. In yet another embodiment, thesecond portion includes a second key and the processor displays on thescreen a third key indicating a location of the second key in the secondside, and when one of the second and third keys is hit, the processorperforms a second function assigned to the second key.

In accordance with another aspect of the present invention, a method forpresenting a virtual keyboard comprising a plurality of keys on a touchscreen device comprising a screen and touch-sensitive elements, a firstportion of the touch-sensitive elements being arranged inside the screenand a second portion of the touch-sensitive elements is arranged outsideof the screen is disclosed. The method comprises presenting the virtualkeyboard, wherein all of the virtual keyboard is displayed on the screenand falls on the first portion of the touch-sensitive elements; movingthe virtual keyboard, in response to a user input, to a position inwhich some keys including a first key of the virtual keyboard arepartially displayed on the screen and are visible, the visible portionsof those partially displayed keys fall on the first portion of thetouch-sensitive elements and invisible portions fall on the secondportion of the touch-sensitive elements, the first key covers a firstportion of the first portion of the touch-sensitive elements and a firstportion of the second portion of the touch-sensitive elements; and themethod further comprises when the first key is hit, performing aparticular function assigned to the first key.

In one embodiment, the performing step is performed if a hit on one ofthe first portion of said first portion of the touch-sensitive elementsand the first portion of the second portion of the touch-sensitiveelements is detected.

In one embodiment, even if the first key is completely falls inside afirst portion of the second portion and is completely invisible, when ahit on the first portion of the second portion of the touch-sensitiveelements is detected, the performing step is performed.

In yet another embodiment, the screen is located on a first side of thetouch screen device, the second portion of the touch-sensitive elementsis located on a second side of the touch screen device other than thefirst side, the second portion of the touch-sensitive elements include asecond key, and the method further comprises displaying on the screen athird key indicating a location of the second key in the second side.The method further comprises when the second key is hit, performing asecond function assigned to the second key. In one embodiment, themethod further comprises when the third key is hit, performing thesecond function.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned and other features and advantages of this invention,and the manner of attaining them, will become more apparent and theinvention will be better understood by reference to the followingdescription of exemplary embodiments of the invention taken inconjunction with the accompanying drawings, wherein:

FIG. 1 shows a block diagram of a touch screen device according to anexemplary embodiment of the present invention;

FIG. 2 shows exemplary components of display 120 of the touch screendevice shown in FIG. 1;

FIG. 3 shows a different view of the exemplary components of display 120of the touch screen device shown in FIG. 1;

FIG. 4 shows an exemplary virtual keyboard in which some keys arepartially visible but still can be activated by touching the invisibleparts;

FIG. 5 shows an exemplary process of using the exemplary keyboard shownin FIG. 4;

FIG. 6 shows an exemplary process of dynamically adapting a virtualkeyboard to match the signature representing a user;

FIG. 7 shows an exemplary virtual keyboard before being adapted to matchthe signature representing a user;

FIG. 8 shows the virtual keyboard shown in FIG. 7 has been adaptedaccording to a signature representing user with bigger hands;

FIG. 9 shows the virtual keyboard shown in FIG. 7 has been adaptedaccording to a signature representing user with smaller hands;

FIG. 10 shows an exemplary gesture to increase depth of keys in avirtual keyboard;

FIG. 11 shows an exemplary footprint of a key in a virtual keyboard;

FIG. 12 shows that an example that a hit may not be centered at theintended key;

FIG. 13 shows an exemplary adapted virtual keyboard to fix the mismatchproblem shown in FIG. 12;

FIG. 14 shows an exemplary key in a virtual keyboard and its footprintwith respect to the matrix of the touch-sensitive elements;

FIG. 15 shows different sizes and locations of footprints on differentkeys in a virtual keyboard;

FIG. 16 shows that a key in a virtual keyboard can be dynamicallyadjusted based on the detected hit footprint;

FIG. 17 shows another exemplary adapted virtual keyboard to fix themismatch problem shown in FIG. 12;

FIG. 18 shows yet another exemplary adapted virtual keyboard to fix themismatch problem shown in FIG. 12;

FIG. 19 shows an exemplary adapted virtual keyboard, in which the sizesof keys are not the same;

FIGS. 20 and 21 show that the location of a hit footprint of a keydepends on which key was hit prior to the hit to that key;

FIG. 22 shows exemplary hit footprints of the ‘$’ key when the immediatelast lasts are the ‘L’ key and the ‘ä’ key, respectively;

FIG. 23 shows exemplary vector representation for footprints of a key,in which each vector is associated with another key and an adaptedversion of the key;

FIG. 24 shows that the position of a key is dynamically changed whenanother key is hit;

FIG. 25 shows exemplary gestures to move a key to a different locationin a virtual keyboard;

FIG. 26 shows an exemplary process for adding a key to a virtualkeyboard;

FIG. 27 shows an exemplary user interface to aid a user to add a key toa virtual keyboard;

FIG. 28 shows that multiple virtual keyboards can be created by usingthe key adding feature as shown in FIG. 26; and

FIG. 29 shows that virtual keys can be on the back of a touch screendevice;

FIG. 30 shows an exemplary touch-sensitive and tactile surface that canprovide tactile feedback;

FIG. 31 shows an exemplary process that monitors changes of signature;and

FIG. 32 shows that in monitoring changes in dynamic signature, a key canbe divided in five areas.

The exemplifications set out herein illustrate preferred embodiments ofthe invention, and such exemplifications are not to be construed aslimiting the scope of the invention in any manner.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 1, an exemplary, non-limiting aspect of a portablecomputing device (PCD) is shown and is generally designated 1. As shown,the PCD 1 comprises input/output (I/O) means such as I/O block 10,processing means such as processor 20, memory means such as memory 30,and user interface which includes a display 120 and optionally a keypad(not shown) having some hardware keys to complement a virtual keyboard.Although illustrating a PCD as the touch screen device, the principlesof invention can be applied to any apparatus and device, portable ornot, having a display with touch-sensitive screen. The display may beany display, such as LCD, OLED, electronic Ink, and plasmas, and thedisplay may be a part of a TV, a PC, a mobile phone, a tablet, a digitalcamera, a camcorder, and a DVD player. The display may be used with aset-top box and any of the above-mentioned devices. For simplicity andclarity of description, identical elements are referred to usingidentical reference numerals in all figures and certain conventionalelements associated with the PCD 1 such as certain control signals,power signals and/or other elements may not be shown in FIG. 1.

I/O block 10 is operative to perform I/O functions of the PCD 1.According to an exemplary embodiment, I/O block 10 may be operative toreceive signals such as audio, video and/or data signals in analogand/or digital format from a broadcast source such as an Internetsource, and other devices such as a PC, a hard disk connected through,for example, USB, network, or HDMI interfaces. I/O block 10 may also beoperative to output signals to an external device.

Processor 20 is operative to perform various signal processing andcontrol functions of the PCD 1. According to an exemplary embodiment,processor 20 processes the audio, video and/or data signals providedfrom I/O block 10 by performing functions including tuning,demodulation, forward error correction, transport processing, ifnecessary, and decompressing functions to thereby generate digital datarepresenting audio, video and/or data content. The digital data producedfrom such processing functions may be provided for further processingand/or output to the display 120.

Processor 20 is also operative to execute software code that facilitatesand enables performance of the various embodiments and methods of thepresent invention described herein. Further details regarding theseaspects of the present invention will be provided later herein.Processor 20 is also operative to perform and/or enable other functionsof the PCD 1 including, but not limited to, executing programs stored inthe memory 30, processing user inputs made via touching thetouch-sensitive elements located inside or outside the area of thescreen of the display 120, and user inputs from other user input devices(not shown), administering one or more timers, enabling on-screendisplays, reading and writing data from and to memory 30, and/or otheroperations. Processor 20 may be embodied using one or more integratedcircuits (ICs).

According to the principles of the invention, the processor 20 may beoperative to detect touching (or a touch) on the area havingtouch-sensitive elements; display a soft or virtual keyboard on thedisplay 120 in response to a user signal such as touching or hitting apredefined position in the area having touch-sensitive elements orplacing figures on the screen of the display 120 in a predeterminedmanner. More details will follow.

Memory 30 is operatively coupled to processor 20 and performs datastorage functions of the PCD 1. Depending on implementation, at least aportion of memory 30 may be included on the same IC(s) as processor 20.According to an exemplary embodiment, memory 30 stores data including,but not limited to, software code, on-screen display (e.g., menu,virtual keyboard) data, user selection/setup data, a signaturerepresenting a user, a hit footprint of a key of a virtual keyboard,and/or other data. The memory 30 may also include a hard disk drive(HDD), or DVD drive, PROM, SRAM, or combination thereof.

The screen of the display 120 may cover an entire touch-sensitive areaor a portion of a touch-sensitive area. In other words, thetouch-sensitive area may extend beyond the area of the screen for thedisplay 120. The touch-sensitive area includes conventionaltouch-sensitive elements. The processor 20 is operative to detect whichelements are touched in a conventional manner.

FIGS. 2 and 3 depict diagrams of exemplary components of display 120 ofthe PCD 1. As shown, display 120 may include a light source 300, ascreen 310, and a sensing layer 320. The sensing layer 320 includestouch-sensitive elements and as pointed out above, may extend beyond thescreen 310 and may not cover all the area of the screen 310.

Light source 300 may include a mechanism (e.g., a backlight) thatprovides backlighting to a lower surface of screen 310 in order todisplay information. For example, light source 300 may include one ormore incandescent light bulbs, one or more light-emitting diodes (LEDs),an electroluminescent panel (ELP), one or more cold cathode fluorescentlamps (CCFL), one or more hot cathode fluorescent lamps (HCFL), etc.that illuminate portions of screen 310. Incandescent light bulbs may beused when very high brightness is desired. LEDs may be used in small,inexpensive lighting arrangements, and may include colored or whitelight. An ELP may be used for larger lighting arrangements or when evenlighting is desired, and may be either colored or white. CCFLs may beused in large lighting arrangements and may be white in color. Inanother example, light source 300 may employ one or more diffusers orlight guides to provide even lighting from an uneven source. In stillanother example, light source 300 can include any color light source(e.g., yellow, green, blue, white, etc.) or any combination ofcolored/non-colored light sources. The light provided by light source300 may also be used to provide front lighting to an upper surface ofscreen 310 that faces a user. In case that the display is of a type ofliquid ink, OLED or any other type, that do not need a backlightingsource, light source 300 can be omitted.

Screen 310 may include any mechanism capable of providing visualinformation (e.g., text, images, video, incoming or outgoing calls,games, phone books, the current time, emails, etc.) to a user. Forexample, screen 310 may include a liquid crystal display (LCD), such asa thin film transistor (TFT) LCD, etc. In one exemplary implementation,screen 310 may include a plastic substrate that arranges TFT on a metalfoil (rather than on glass), which may permit screen 310 to recover itsoriginal shape after being bent. Screen 310 may include a color filtercoated onto the plastic substrate, which may permit screen 310 todisplay color images. In other implementations, screen 310 may include amonochrome, flexible LCD.

In one implementation, screen 310 may include any number of color and/ormonochrome pixels. In another implementation, screen 310 may include apassive-matrix structure or an active-matrix structure. In a furtherimplementation, if screen 310 is a color array, each pixel may bedivided into three cells, or subpixels, which may be colored red, green,and blue by additional filters (e.g., pigment filters, dye filters,metal oxide filters, etc.). Each subpixel may be controlledindependently to yield numerous possible colors for each pixel. In otherimplementations, each pixel of screen 310 may include more or less thanthree subpixels of various colors other than red, green, and blue.

Sensing layer 320 may include touch-sensitive elements that detect thepresence of a user's finger 330 (e.g., a thumb, an index finger, amiddle finger, a ring finger, or a pinkie finger) on display 120, detectthe location (or touch area) of finger 330 on display 120, determineshow many fingers a user has on display 120, etc. For example, sensinglayer 320 may include a layer of capacitive material (e.g., providedunder a protective covering (not shown)) that may experience a change inelectrical charges (e.g., a change in the amount of charge stored) whenfinger 330 contacts sensing layer 320. In one exemplary implementation,sensing layer 320 may include self capacitance circuitry that includesan array of electrodes and monitors changes in the array of electrodeswhen a user contacts sensing layer 320 (e.g., with finger 330). Inanother exemplary implementation, as shown in FIG. 3, sensing layer 320may include a layer of driving lines 340 that carry current, and aseparate layer of sensing lines 350 that detect changes in electricalcharge when a user contacts sensing layer 320 (e.g., with finger 330).

Sensing layer 320 may sense a change associated with its electricalproperties every time a user contacts sensing layer 320, and may providethis information to processor 20 and/or memory 30. Processor 20 mayutilize this information to determine a shape, a size, and/or a locationof a user's finger (or fingers) on display 120. In one exemplaryimplementation, processor 20 may calculate touch area(s) associated witha user's finger(s) based on information received from sensing layer 320,and may reconfigure display element(s) (e.g., keys, icons, etc.)associated with display 120 based on the calculated touch area(s).

Although FIGS. 2 and 3 show exemplary components of display 120, inother implementations, display 120 may contain fewer, different,differently arranged, or additional components than depicted in FIGS. 2and 3. In still other implementations, one or more components of display120 may perform one or more other tasks described as being performed byone or more other components of display 120.

According the principles of the invention, the sensing layer 320, whichincludes touch-sensitive elements, is larger than the screen 310 ofdisplay 120, and the processor 20 can determine a shape, a size, and/ora location of a user's finger (or fingers) on the sensing layer 320outside of the screen 310. In effect, a first portion of thetouch-sensitive elements is arranged inside the screen 310 (overlappingwith the screen 310) and a second portion of the touch-sensitiveelements is arranged outside the screen 310 (not overlapping with thescreen 310). If a virtual key falls on the first portion of thetouch-sensitive elements, the virtual key is completely visible. If thevirtual key falls on the second portion of the touch-sensitive elements,the virtual key is invisible. If a portion of the virtual key falls onthe first portion of the touch-sensitive elements and remaining portionfalls on the second portion of the touch-sensitive elements, only theportion of the virtual key falls on the first portion of thetouch-sensitive elements is visible. The remaining portion is invisibleand is visualized by a user as in the portion of the second portion ofthe touch-sensitive elements if the screen 310 is extended to cover thatportion of the second portion of the touch-sensitive elements. In thecase that the entire virtual key completely falls on the second portionof the touch-sensitive elements and thus invisible, the imaginaryposition of the invisible key can be inferred from its relationship witha second virtual key. For example, if the invisible key is onecentimeter below the second virtual, the invisible key would still beone centimeter below the second virtual key even if the first virtualkey is not visible. The processor 20 in displaying the virtual keysperforms the same extrapolation, so that the processor 20 can detectthat an invisible key has been hit. Each virtual key may be assigned afunction and when the virtual key is hit (touch and release), theprocessor 20 performs that function. For example, the function may beentering a character to an entry field, such as a text field, or textfile, going to a next page, or any function of a key in a conventionalkeyboard.

When a virtual key is partially visible, a hit on a portion of either orboth of the visible and invisible portions causes the processor 20 toperform the assigned function of that virtual key. When a virtual key isfully visible, a user must hit on a portion of the visible key and if avirtual key is fully invisible, a user must hit the correspondinglocation on the second portion of the touch-sensitive elements in orderto cause the processor 20 to perform the assigned function of thevirtual key.

The virtual keys may be part of a virtual keyboard and some keys, suchas the space bar, of the virtual keyboard can be partially or completelylocated in the second portion of the touch-sensitive elements. Ineffect, those keys are not visible. Large keys, such as the spacebarwould be found easily when below the screen 310 (at the usual position),even if they are invisible. It should be noted that an invisible key canalso be adapted using methods described later herein.

FIG. 4 shows an exemplary virtual keyboard 410 on the display 120, inwhich the sensing layer 320 having touch-sensitive elements extendsbeyond the area of the screen 310. The second portion of touch-sensitiveelements is arranged to be consecutive to the bottom edge of the screen310 and to the first portion of the touch-sensitive elements. In thisexample, the spacebar and its horizontal neighbors are only displayedpartially (upper portion on the screen 310 and on the first portion ofthe touch-sensitive elements, lower portion outside of the screen 310and on the second portion of the touch-sensitive elements) such thattheir positions are easier to find visually. The second portion of thetouch-sensitive elements may exist in any of all sides of the screen310. For example, if the virtual keyboard 410 is moved to the top, therow of the number keys can be partially displayed. Similarly, the keyson the right or left side can be partially displayed if touch-sensitivearea is also extended in that side.

It is noted that the processor 20 can move the keyboard 410 anywhere inthe screen 310 in response to a user input. For example, a user can dragthe keyboard 410 to any position on the screen 310. However, if aninvisible key falls outside of the area for the touch-sensitiveelements, a user will not be able to activate the assigned function ofthat invisible key even if the user hits that invisible key because thehit is not detectable by the processor 20.

FIG. 5 shows an exemplary process 500 of using the exemplary keyboard410. For purposes of example and explanation only, the steps of FIG. 5may be explained hereinafter with specific reference to the exemplaryembodiments of FIGS. 1, 2, 3 and 4 described above. At step 505, theprocessor 20 displays a virtual keyboard, wherein the entire virtualkeyboard is displayed on the screen 310 and falls on the first portionof the touch-sensitive elements in response to a user input.Illustratively, the virtual keyboard is displayed when a user turned onthe PCD 1, or when the processor 20 detects that a user taps the display20 a predetermined number of times, for example twice.

At step 510, the processor 20 moves the virtual keyboard 410, inresponse to a user input, to a position in which some keys including afirst key of the keyboard are partially displayed on the screen 310 andare visible. The visible portions of those partially displayed keys fallon the first portion of the touch-sensitive elements and the invisibleportions of those partially displayed keys fall on the second portion ofthe touch-sensitive elements. An example of the moved keyboard is shownin FIG. 4. In this example, the first key is the space bar, and thevisible portion of the first key falls on the first portion of the firstportion of the touch-sensitive elements and the invisible portion fallson the first portion of the second portion of the touch-sensitiveelements.

The exemplary keyboard includes a plurality of rows, and those partiallydisplayed keys are arranged in one of the rows. In this example, thepartially displayed keys are located in the row that includes the spacebar key as shown in FIG. 4. Although illustrated as rectangle in shape,the keyboard can have a different shape. For example, the keyboard andone of more of its rows can be askew, flexed, of radial shape orscattered.

When the processor 20 detects that the first key has been hit, theprocessor 20 enters a space into an entry field, such as a text field,or a text file at step 515 because the assigned function of a space keyis to enter a space. The first hit can be on the first portion of thefirst portion of the touch-sensitive elements only, on the first portionof the second portion of the touch-sensitive elements only, or on both.

The processor 20 may further move the keyboard to a position in whichthose partially visible keys become completely invisible and fall on thesecond portion of the touch sensitive elements, in response to anotheruser input. In the newly moved keyboard, the first key is invisible andfalls on a second portion of the second portion of the touch-sensitiveelements and when the processor 20 detects a hit on the second portionof the second portion of the touch-sensitive elements, the processor 20enters a space into the entry field, such as a text field, or the textfile.

According to the principles of another invention, a virtual keyboard canbe adapted to fit the width of the user's hands and fingers. FIG. 6illustrates a process 600 of dynamically adapting a virtual keyboard.For purposes of example and explanation only, the steps of FIG. 6 may beexplained hereinafter with specific reference to the exemplaryembodiments of FIGS. 1-3 described above and other figures describedlater.

At step 605, the processor 20 detects sizes and positions of touches asa signature on the screen 310 representing fingers of a user. In oneembodiment, a user places all ten fingers on the screen 310 withoutdisplay of a default keyboard, allowing the processor 20 to build andpresent a matching layout of the displayed virtual keyboard according tothe sizes and spreads of the fingers. The relative positions and sizesof the ten fingers are recorded by the processor 20 as a signature forfuture reference. The keys produced by the processor 20 do not have tobe arranged strictly in an “in-line orientation”. For example thefingers of a hand can be placed in a slight, natural bow or angle andthe hands can be in a more ergonomic position not necessarily strictlyside-by-side.

In another embodiment, since both thumbs handle only the space bar, itwould be sufficient to place only one of the thumbs and therefore 9fingers on the touch screen for the processor 20 to determine the sizeand shape of the virtual keyboard. Further, based on the spaces betweenthe fingers other than the thumb(s), the position of the space bar canbe evaluated even if no thumb is detected by the processor 20.

In yet another embodiment, a default keyboard, for example, theexemplary conventional European keyboard shown on FIG. 7 is displayed onthe screen 310. Illustratively, the virtual keyboard 710 is displayedwhen a user turned on the PCD 1, or when the processor 20 detects that auser taps the screen 310 a predetermined number of times, for exampletwice. A user then naturally and comfortably places the pinkie, ring,middle, and index fingers on the left hand 430 on ‘A’, ‘S’, ‘D’, and ‘F’keys, respectively, and the pinkie, ring, middle, and index keys of theright hand 420 on the “ó/é”, ‘L’, ‘K’, and ‘J’ keys, respectively.However, the user needs only aligning the index finger of the right hand420 with the ‘J’ key or the index finger of the left hand 430 with the‘F’ key. If neither index finger is aligned with the respective key, theprocessor 20 should warn the user to adjust the positions. Since thefingers are placed naturally, the other three fingers of each hand maynot be aligned with the corresponding three keys. The relative positionsand sizes of the fingers are stored as the signature of the user.

The use of eight, nine, or ten fingers as a signature is exemplary.Using four or more fingers with at least one finger aligning to aparticular key may be sufficient if a virtual keyboard is displayed bythe processor 20.

In yet another embodiment, the processor 20 stores user identification,such as a login name, in addition to the corresponding virtual keyboardand a signature. In this embodiment, the processor 20, in response to auser signal for initialization, displays a default virtual keyboard anda login field, so that the user can use the default virtual keyboard toenter the user identification in the login field. Once the processor 20has received the user identification, the processor 20 can find whetherthere is a corresponding signature and hence the corresponding virtualkeyboard in the memory 30.

In addition to entering the login name, a user may be required to entera password as well. Other identification of a user may include anythingbiometrical that uniquely identifies a user, and any combination of apassword, a key, and biometrical identifier.

At step 610, the processor 20 checks whether there is a correspondingvirtual keyboard associated with the detected signature in the memory30. If the corresponding virtual keyboard exists in the memory 30, theprocessor 20 at step 615 retrieves the corresponding virtual keyboardfrom the memory 30 and displays the corresponding virtual keyboard onthe screen 310. This corresponding virtual keyboard should be an adaptedkeyboard produced previously with respect to this particular signature.If the default virtual keyboard 710 is displayed, the default virtualkeyboard 710 is replaced by the corresponding virtual keyboard. Text 440in FIG. 7 is entered by a user through the use of the adapted virtualkey board.

If no corresponding virtual keyboard exists in the memory 30, theprocessor 20 at step 620 produces a first virtual keyboard matching thedetected signature and displays the first virtual keyboard, so that theuser can start using the keyboard to enter data. In effect, the firstvirtual keyboard is an adapted virtual keyboard that matches thedetected signature, i.e., the relative positions and sizes of thosefingers detected. The processor at step 680 then stores the adaptedkeyboard in the memory 30 as corresponding to the detected signature.

As stated above, if the corresponding virtual keyboard does not exist,the processor 20 produces a first virtual keyboard matching the detectedsignature, so that a hit to a key from the user more likely produces ahit having its geometric center matching the geometric center of thatkey, and thus reducing the possibility of a mishit.

For example, if a default virtual keyboard, such as the default virtualkeyboard 710 in FIG. 7, is displayed, the processor 20 adjusts thedefault virtual keyboard 710 to a wider and larger virtual keyboard 810as the first virtual keyboard, as shown in FIG. 8 for bigger hands. Forsmaller hands, the processor 20 may shorten the virtual keyboard 710 toa narrower and smaller virtual keyboard 910, as shown in FIG. 9. Ineffect, the processor 20 adapts the default virtual keyboard 710 toproduce the first virtual keyboard.

As compared to the virtual keyboard 710, the width of each key in thevirtual keyboard 810 is larger and the width of each key in the virtualkeyboard 910 is smaller. The processor 20 may also automatically changethe depth of a key proportional to the change in width (aspect ratio ofa key remain the same) of that key.

As an alternative, the processor 20 does not adjust the depthautomatically but as instructed by the user. To manually adjust thedepth, a user may place at least one finger, for example, the indexfinger of the right hand 420, on a virtual key, for example, the ‘U’key, in an upper row, while maintaining the positions of the other sevenfingers, as shown in FIG. 10, and in response, the processor 20increases the depth of each key. If the index finger is placed in alower row, the processor 20 decreases the depth of each key. The widthof a key depends on the size of the finger (larger fingers will need awider keyboard) but the depth of a key depends on the length of the topsegment of a finger but also depends on the habit of the user. Forexample, if a user uses finger tips most of the time, the depth of a keycan be smaller. The processor 20 may store the manually adjustedkeyboard, when the user releases all the fingers from the screen 310signaling that the initialization process has been completed.

In another embodiment, when the user spreads the fingers (more distancebetween fingers), the processor 20 proportionally increases the depth ofthe virtual keyboard and when the user closes the fingers (opposite ofspreading) the processor 20 decrease the depth of the keyboard. When theuser sees that the depth is appropriate, the user stops moving thefinger and the processor 20 stops changing the depth.

As mentioned earlier, a user can drag a virtual keyboard to any positionin the screen 310. This feature is advantageous particularly on largerscreens when the keyboard covers part of the application or a “window”(that might arise during the use or by user input). In this situation,the user can quickly move the keyboard to a new position almost withoutinterrupting the typing. If the user moves the keyboard over the edge ofthe touch screen and beyond the touch-sensitive area, the process or 20may warn the user aurally or visually. In response, the user may shrinkthe virtual keyboard to keep the virtual keyboard within the touchscreen or touch-sensitive area. The user may also ignore the warning andcontinue to move the keyboard beyond the touch screen edge till finallythe virtual keyboard disappears.

In another embodiment, which is more suitable for the “hunt and peck”users, a user places as few fingers as possible, for example, the twoindex fingers, on the screen 310 for initial set-up by the processor 20.However, since the other fingers and the thumbs are not present, theprocessor 20 produces only “in-line orientated” virtual keyboard. Thedistance between the index fingers can be used by the processor 20 as areference for evaluating the size and position of the keys.

After the initiation process, the processor 20 can dynamicallyadjust/adapt the layout/size/position of keys in the displayed keyboardduring a user editing session or training session according to theprinciples of the invention. In one embodiment, the user can place thefingers used for initiation of the keyboard again on the touch screenand move, spread/narrow or rotate the fingers/hand to adjust thekeyboard. The processor 20 detects the multi-finger gesture and thefinger movement (sliding in the screen plane) to trim the size andposition of each virtual key. This slight finger movement would define aregion on the screen slightly larger than a fingertip to be therespective virtual key.

During the use of the first virtual keyboard, the processor 20 maycollect a hit statistics for a key, so that the processor 20 may furtheradjust at least one of the size, shape, and position of the key to matchthe typing habit of a user. A disadvantage of using a virtual keyboardis that a user never knows, whether he has hit the key in the middle, atthe border or not at all. As such, hits on the same key may be ondifferent positions. For example, the hits on the ‘U’ key in FIG. 11 maybe centered on a position slight above and to the right of the geometriccenter of the ‘U’ key. This problem occurs more often for those keys onthe far right or far left side, for example the “$ (£)” key, as shown inFIG. 12.

The activation of a virtual key is usually triggered upon release of ahit. A hit, however, may cover an area (fingerprint) larger than theintended virtual key. To more accurately detect which virtual key ishit, the processor 20 may compute the geometric center of a fingerprintand determine that a key is hit if the geometric center of thefingerprint falls within that key. In FIG. 11, the detected centers arelabelled with letter ‘u’. The calculation of the centers isconventional. Other methods of matching a fingerprint with a key arewell known and can be used with the principles of the invention.

The processor 20 automatically detects this variation and evaluates theregion in which the keyboard should be receptive for each key. Theprocessor 20 in effect adapts the individual keys to the user habits inrespect of position, size and shape while a user hitting the keys toreduce the error rate of typing by a user on a virtual keyboard. Theadaptation can take place all the time when the keyboard is in use oronly during a learning phase of the system. However, the adaptation ispreferably linked to a given user, so that each user has his/herpersonal parameter set that is loaded initially and then adjusteddepending on the current setting of the adaptation. This set ofparameters can be selected either by an input (user identification,signature, etc.) of a user or by recognizing the typical fingerposition. The user may also signal the processor 20 to store the newlyadapted virtual keyboard as the corresponding virtual keyboard for thesignature.

As an example, after collecting the hit statistics for the “$ (£)” key,the processor 20 detects the mismatch between the geometric center ofthe “$ (£)” key and the average geometric center of the hit statistics,as shown in FIG. 12 and moves the “$ (£)” key to the right to align withthe average geometric center of the hit statistics, as shown in FIG. 13.The two centers should be on the same imagined vertical line but may notmatch each other. In an alternative, the processor 20 can collect thehit footprint of the “$ (£)” key, and shift the geometric center of the“$ (£)” key to the geometric center of the hit footprint. A hitfootprint of a key is the collection of all the touch-sensitive elementsthat are touched from a predetermined number of hits on that key.

FIG. 14 illustrates the detected hit footprint 1220 of a key. From thefootprint 1220, a geographic center can be calculated.

Furthermore, the user may hit some keys more consistently than others.For example, as shown in FIG. 15, the user hits the ‘Z’ key moreconsistently than the ‘H’, ‘8’, ‘9’, and ‘U’ keys, and the area of thehit footprint of the ‘Z’ key therefore is smaller than hit footprints ofthe others. The processor 20 can also determine from a hit footprint inFIG. 14 a shape and a size of the key 1420, so that the key 1420 becomesmore suitable for the user. Illustratively, each grid includes a touchsensitive element in FIG. 14. An illustrative key 1420 covers 13×13grids. The shape can be formed by connecting peripheral points in thehit footprint 1440. It is noted that a guard band 1430, which is theregion between two keys, can be as small as the distance between twoneighboring touch sensitive elements.

Although for simplicity, FIG. 14 shows that each hit activates only onetouch-sensitive element, more than one touch-sensitive element isactivated in practice.

According to the footprints collected, the processor 20 resizes the keysaccordingly, as shown in FIG. 15. The user hits the ‘Z’ key moreconsistently and the processor 20 shrinks the ‘Z’ key making more roomfor other keys that may be enlarged. By contrast, the ‘H’ key isadjusted larger. The processor 20 may also change the shape of a keyaccording to the hit footprint for that key. For example, the number ‘8’and ‘9’ keys are adjusted to have a five-sided shape.

FIG. 16 illustrates that the shape and size of a key can be dynamicallyadjusted based on the detected hit footprint. The X and Y axes definethe touch area 1620 representing a key, which illustratively is a 13×13grid area. The Z axis indicates number of hits/strokes. FIG. 16A showsthat no hit has been detected in the area 1620. FIGS. 16B and 16D showthat hits are confined to smaller and larger circle areas, respectively.The smaller circle is located approximately at the geometric center ofthe area 1620. Thus, the size of the key may be reduced. The biggercircle in FIG. 16D, however, is not located approximately at thegeometric center of the area 1620 and tilt to the top-right. Thus, theposition of the key should be moved toward the top-right, such that thegeometric center of the area 1620 matches the geometric center of thebigger circle. Since the size of the bigger circle is smaller than thesize of the area 1620, the key can also be made smaller. FIGS. 16C and16E show that the hits are confined to smaller and larger squares,respectively. The geometric center of the smaller square in FIG. 16Cmatches the geometric center of the area 1620. Two geometric centers aresaid to be matched if the distance between the two centers is less thana predefined number of grids, such as one grid, which may be set by auser. The key therefore needs not be moved but can be made smaller. Thebigger square in FIG. 16E takes up the entire area 1620 and beyond. Assuch, the key should be enlarged. FIG. 16F shows that the hits areconfined to a triangular area, i.e., the hit pattern is asymmetric. Thearea 1620 should be moved such that the two geographic centers match.

In addition to changing size and the location, the processor 20 maychange the shape of a key as well. For example, the keys in FIGS. 16Band D can be reshaped to a circular, the keys in FIGS. 16 C and E can bereshaped to a rectangle, and the key in FIG. 16F can be reshaped to atriangular. As pointed out previously, a shape can be formed byconnecting peripheral points in the hit footprint (the connected shape).In one embodiment, if a predefined number of shapes are used, theprocessor 20 may use one of the predefined shapes, which has the minimumerror when enclosing the connected shape, as the adapted shape. Theerror can be calculated, for example, by counting the number oftouch-sensitive elements (grids in FIG. 16) between an enclosingpredefined shape and the connected shape.

If the deviation is too big that the desired key is not hit at all and aneighboring key may be activated, the user has to use the backspace keyto delete this letter and enter the correct one—respectively adictionary function makes a correction proposal (as it is nowadays).According to the principles of the invention, the processor 20 maydetect a miss hit (stroke) and add the miss hit to the footprint of thedesired key. For example, if the hit misses the desired key completelyand touches a neighboring key, and the user uses the backspace key todelete this letter and enter the correct one, the processor 20 canidentify the desired key and attribute the miss hit as part of thefootprint for the desired key, so that the location, the size, and theshape of the desired key can be correctly adjusted.

For another example, the processor 20 may base on an automaticcorrection by the editing software to identify the desired key and themiss hit for more accurately detecting the footprint of the desired key.Thus, by monitoring the input and recognizing such a deleting/correctingsituation, the processor 20 can use this information to influence theadaptation of the keyboard. For example, if a wrong letter is replacedmultiple times with the same correction, the processor 20 can adjust theat least one of sizes, shapes, and positions of keys to reduce the rateof wrong entries. An error checking dictionary can be used for detectingmiss hits and adjusting the keyboard layout continuously as well. Theuse of a dictionary shifts the error detection from a single lettertyping error to a word based version. This means that the processor 20can also check if the typographical is based on hitting a neighbor keyor just an ordinary mistake.

FIGS. 17 and 18 show two examples of momentary keyboard layouts. FIG. 17shows a partially spread layout spreading only those keys that areresponsible by the pinkie finger of the right hand 420, for example,from the ‘0’ key to the right and below to fix the problem of the userstraddling the pinkie finger of the right hand 420 more than expected asshown in FIG. 12. As such, the processor 20 partially spreads thosekeys, including number ‘0’ key, that the pinkie finger of the right hand420 is responsible. FIG. 18 illustrates another solution to the sameproblem, in which the processor 20 simply widens the “£/$” key andshrinking the ‘RTN’ key accordingly. FIG. 19 shows the complete layoutafter a learning phase with many keys changed in size and position.Notice that the keys do not have the same size.

The processor 20 may store this newly adapted keyboard as thecorresponding virtual keyboard for the detected footprint automaticallyor in response to a user input. Since the changes are dynamic, the usermay not want to save the newly adapted virtual keyboard as thecorresponding virtual keyboard until the user feels that the newlyadapted keyboard is satisfactory or until the user has completed theediting session.

According to the principle of the invention, when running such anadaptive system it may be helpful for the user to get some feedbackabout the systems state. It may be helpful to warn the user with anincreasing sound (other feedback like light, vibration, or a tactilefeedback system is possible) that the processor 20 is unable to adjustthe keyboard and will initialize the keyboard to a default basic layout.The warning may start with “please type more precise” and later becomes“please reinitialize.” These warning messages can be in audio or intext. The processor 20 may provide a haptical feedback, just as aresponse to a touch on the screen, at the same time. A tactile feedbacksystem, in which the processor 20 rises/lowers the surface, gives theuser the haptic feeling of pressing real buttons. In one embodiment,this tactile feedback system could be used to give the user a hapticfeeling of touching the edge of the key. See FIG. 30 as an example. Withthe feedback, the user can adjust the position of a hit and hit moreprecisely.

The inventors observed that the position where a key is hit may bedepending on the immediately previous key that was hit. For example,FIG. 20 depicts the situation where the ‘$’ key is hit after an key. Thepinkie finger is straddled wide. By contrast, as shown in FIG. 21, ifthe previous key hit is an ‘ä’ (pressed by the pinkie finger too), thepinkie finger has to move only one key and thus produces a different hitfootprint, as shown in FIG. 22. Thus, the inventors recognized that anadaptive keyboard can be further improved by having different keylayouts that are given by the previously hit key. The processor 20 thusmay store this association in the memory 30. For each key, the number ofassociations in the memory 30 may be up to the number of the other keysthat are responsive by the same hand. For example, the processor 20collects the footprint of a second key from a predefined number of hitsto the second key, all of the hits immediately following a hit to afirst key, at least repositions or resizes or reshapes the second key(the adapted second key) according to the collected footprint, computesthe relative distance from the geometric center of the adapted secondkey to the geometric center of the first key, and stores the adaptedsecond key, the relative position information with respect to the firstkey and an identification of the first key in the memory 30 indicatingthat the adapted second key is associated with the first key. When thefirst key is hit in the future, the processor 20 replaces the second keywith the adapted second key in the memory 30 and positions the adaptedsecond key in the position information retrieved from the memory 30. Theprocessor 20 repeats the same for other adapted keys in the memory thatare associated with the first key. Advantageously, for each key hit, theprocessor 20 presents a different keyboard with possibly all the keysadjusted or adapted to this key.

For this novel approach, a position of a first key may also be definedas a value from which the position of the first key can be derived fromthe geometric center of an immediately last hit key. In effect, thevalue allows determining the position of the first key from thegeometric center of the last immediately last hit key. An example ofsuch a value is a vector with the starting point at the geometric centerof an immediately last hit key and the ending point at the geometriccenter of an adapted version of the first key. The adapted version ofthe first key with respect to a particular immediately last hit key isalso stored in the memory 30 as associated with the associated vector.The associated vector indicates the position of said adapted first keyrelative to the immediately last hit key. Thus, by knowing theimmediately last hit key, the processor 20 can retrieve the adaptedversion of the first key and display that adapted version of the firstkey in a position defined by the associated vector. A vector can berepresented by length-angle pair in a polar system or the X-Y pair in aCartesian system. The adapted version is derived from the footprint ofthe first key with respect to the immediately last hit key. The adaptedversion may only have one of the position, the size and shape changed,or any two of them, or all of them.

Each key now has a set of such values computed by the processor 20,which are vectors in this example, indicating the relative positions ofthe adapted versions of the immediately next hit keys, each setassociating with one of all other keys or, to save computing power ofthe processor 20, the other keys that are responsible by the same hand.See FIG. 23 as an example. To help the readers better understand thevectors, the shapes of footprints are assumed to have the same squareshape in FIGS. 23 and 24, the sizes of the adapted versions are notchanged, and all the keys in the upper row are marked only with a ‘+’ atthe respective geometric centers. The geometric center of a differentfootprint is placed at the end (the arrow head) of a vector as shown inFIGS. 23 and 24. FIG. 24 further illustrates that when the position of ahit to the immediately last hit key changes, the geometric center of thefootprint of the current hit key is changed as well. However, theassociation in the memory 30 needs not be changed because the positionof the current hit key is determined from the relative positioninformation in the memory 30 with respect to each immediately last hitkey.

The set of vectors stored in the memory 30 may be considered a part of asignature of a user. It is a dynamic signature because it cannot becollected until a user starts an editing session and using the virtualkeyboard by hitting keys.

According to another aspect of the invention, the processor 20 enables auser to add, move, and replace keys to fit the user's special needs. Forcertain symbols or special letters it may be desirable to place them onthe top level of the virtual keyboard, so that the user has no need toinvoke the symbol adding feature or press shift-, ctrl-, or alt-key toenter those symbols and letters. As an example, an American user wouldmost probably use an American keyboard, but if the user has to usefrequently the “Euro-sign” (

), the user would be glad to have direct access to this key instead ofusing the tedious procedure to add this symbol. By adding a new key withthe desired symbol or replacing the symbol of an existing key accordingto the principles of the invention, the user can easily handle thatsituation. Instead of a single letter, the processor 20 may allow a textstring like “Dear sirs”, “Yours sincerely” to be assigned to a singlekey. Thus, the processor 20 allows a user to rearrange the keys directlywhen the keyboard is displayed. For example, the processor 20 allows auser to drag and drop the keys to the new position, to delete an unusedkey, to replace a key or to add a key.

In one embodiment, when the processor 20 detects two finger gestures,one from a first key to a second key, and the other from the second keyto the first key, the processor 20 swaps the first and second keys. Forexample, in FIG. 25, the first and second keys are the ‘8’ key and the‘9’ key, respectively, and the processor 20 exchanges positions of thetwo keys. In another embodiment, the processor 20 allows a user to draga key using one finger to a desired position. When the processor 20drops off the key to the desired position, the processor 20 moves thekey originally located in the desired position to the location where thedragged key was located. In effect, the processor 20 exchanges thepositions of the two keys.

In yet another embodiment, when the processor 20 drops off the draggedkey, for example, the ‘5’ key, in the desired position, for examplewhere the ‘2’ key is located, in the same row, the processor 20 shiftsthe affected key and those keys positioned between the affected key andthe dragged key. In this example of FIG. 25, the ‘2’, ‘3’, and ‘4’ areshifted to the right one position and the ‘5’ key then takes up theoriginal position of the ‘2’ key.

FIG. 26 illustrates the process 2600 of adding keys. At step 2605, theprocessor 20 displays a virtual keyboard 2710, as shown in FIG. 27. Thedisplayed virtual keyboard may be the one that the user is currentlyusing or invoked during the initialization process as described above.At step 2610, the processor 20 enables a user to add a first key to thevirtual keyboard 2710 to form a first virtual keyboard. To enable theuser to add the first key, the processor 20 may display a conventionalsymbol table 2750, as shown in FIG. 27, in response to a command fromthe user to add a symbol. As well known, editing software normallyprovides a symbol table, so that a user can select a symbol from thetable and enter that selected symbol into the edited document or field.The processor 20 thus enables the user to assign the added key thefunction of entering the selected symbols. The processor 20 may alsoenable the user to select a size of the key to be added. Illustratively,the processor 20 provides three different size boxes 2760, 2770, and2780, as shown in FIG. 27 for user selection. This may occur in atraining mode, which can be invoked by a user by touching a predefinedposition on the touch sensitive area or touching a position in the touchsensitive area for a predefined number of times. To add a symbol, a usershould first touch one of the size boxes 2760, 2770, and 2780, and thenselects a symbol from the table 2750, as shown in FIG. 27. If no sizebox is selected, the processor 20 should select one of them, forexample, the size box 2760, as a default. The selected symbol ispreferably displayed in the selected size box. The user can then dragthe selected size box with the selected symbol displayed in it to adesired position. The processor 20 may automatically adjust the fontsize of a text string displayed with a new key or abbreviate the textstring or both to fit the space allocated for that new key.

In order to facilitate the situation where the added key is assigned afunction other than entering a symbol in the symbol table 2750, theprocessor 20 may also provide a text box 2790. To add a key representinga text string, a user first selects one of the size boxes, enters thetext string in the text box 2790 and drags the selected size box to adesired position. In the exemplary virtual keyboard 2710 having anAmerican style layout, the “$ (£)” key, the “

” key, and the “

” key are added. Thus, a key can now hold even foreign letters, textfragments, symbols, pictures, graphics, or a multimedia function (sound,video or an action) that can be used/initiated by a user hitting thatadded key in that keyboard. To provide multimedia function, theprocessor 20 should provide an interface for a user to add a key with amultimedia function. For example, a multimedia box is displayed, andwhen the multimedia box is touched by the user, the processor 20provides a browsing function for the user to select a multimedia file tobe associated with that key. The selected file name is entered to thetext box 2790. In effect, the text box (field) 2790 can be used to entera text string, a graphics, a symbol not in the symbol table 2750, thefile name of an audio file, and the file name of a video file. When theuser signals the end of a key adding session by, for example, closingthe symbol table, the processor 20 at step 2615 stores the new virtualkeyboard including the added keys in a memory to be associated with asignature of said user.

The processor 20 detects the signature of the user as described abovewith respect to the initialization process. If the added key is hit, theprocessor 20 will perform its assigned function, such as entering asymbol, entering a string, playing back an audio, or playing back avideo.

Using the key adding feature, a user is able to cause the processor 20to display two virtual keyboards, such as keyboards 2810 and 2895 shownin FIG. 28. The keys in the secondary keyboard 2895 can be added asdescribed above. The secondary keyboard 2895 may be used by the user ora different user for some purposes, such as educational, gaming or othermulti-user purposes.

It should be noted that like other existing keys in the virtualkeyboard, the added keys can also be dynamically adapted using theexemplary process as discussed above with respect to FIG. 6.

During an editing session, the user might step back to theinitialization process by providing user signals as described above withrespect to the initialization process.

In another embodiment, the virtual keyboard can be displayedtransparent, allowing a user continuing to see the displayed contentwhile typing, as shown in FIG. 4. The degree of transparency may varyover the keyboard in relation to the use, importance or interactivity ofthe keys such that the user has a better overview over the wholedocket/screen. This transparency leads to a larger area useable for theobject to be written in.

In yet another embodiment, only part of the virtual keyboard isdisplayed transparent. For example, the row for the number keys in FIG.4 is displayed transparent, but other parts of the virtual keyboard 410are not.

In yet another embodiment, the virtual keyboard is transparent but whena user starts striking/hitting the keyboard, the virtual keyboardbecomes opaque and if the user does not stroke for a predefined time,the virtual keyboard becomes transparent again.

Future tablet-PCs may have buttons or a touch sensitive area on the rearside or edges such that the user does not have to move the grip forcertain actions. Those keys are activated by touching certain predefinedarea located on the rear side or edges, which contains touch-sensitiveelements. The touch of those keys is detectable by the processor 20.This is an example that the entire second portion or some portion of thesecond portion of the touch-sensitive elements is not arranged in aconsecutive manner with the first portion of the touch-sensitiveelements and the screen 310. By contrast, the first portion and theentire second portion of the touch-sensitive elements in FIG. 4 arearranged in a consecutive manner.

These rear virtual keys can be displayed on the screen 310 in thefront-side (fully, transparent or only when interaction is demanded) toshow their corresponding positions on the rear side, but a user muststill touch the corresponding areas in the rear side. Touching thedisplayed keys on the screen 310 produces no effect. These keys canstill be adaptive as the keys in the virtual keyboard in the front sideand can be saved as part of the corresponding virtual keyboard in thememory 30. For example, as shown in FIG. 29, in addition to the keys onthe front side, five keys located on the rear side are also displayed:‘F1’, ‘F2’, ‘F3’, ‘F4’, and ‘F5’. After adaptation, at least one of theshapes, sizes and locations of the five keys may change.

In a further embodiment, the virtual keys from the rear side can bebrought to the front side by pressing an activating key or group of keyslike “CTRL” and “Shift” keys. This will allow the user to activate “rearside keys” from the front side from corresponding virtual keys on thedisplay 120. In a preferred embodiment, this change is visualized bychanging, for example, the color of the corresponding virtual keys onthe display 120.

To move a virtual keyboard to a different location, a user may keep themultiple fingers, for example, three or four fingers from either one orboth hands, on the keys for a predefined time, for example 2 seconds.When the processor 20 detects such a situation, the processor 20recognizes that the user wants to move the keyboard to a differentlocation. When the processor 20 detects that the fingers start sliding,the processor 20 moves the virtual keyboard accordingly.

To quickly make the keyboard disappeared; the user should make a swift(wipe) movement. To get the keyboard back, the user simply places thefingers on the display as previously described.

As discussed in FIG. 6, once the processor 20 detects a signature with acorresponding keyboard, the processor 20 retrieves the correspondingkeyboard for the user to use. As mentioned previously, the set ofvalues, which are vectors in this example, stored in the memory 30 maybe considered a part of a dynamic signature of a user. According to theprinciples of the invention, during an editing session, the processor 20can detect if there is a significant change to the stored dynamicsignature and if there is, the processor 20 can inform the user of sucha change. FIG. 31 illustrates such a process 3100. At step 3105, theprocessor 20 displays a corresponding virtual keyboard according, forexample, the process 600, as described in FIG. 6. The displayed virtualkeyboard should be the last adapted virtual keyboard. At step 3110, theprocessor 20 adapts the displayed virtual keyboard during the editingsession according, for example, the processes described previously.Briefly, the step 3110 includes a step 3114 for detecting key strokes(hits), a step 3116 for determining adapting parameters (such as arelative position represented by, for example, a vector as describedpreviously) and the processor 20 adapts the keyboard accordingly, a stepof saving the determined parameters.

Before saving the determined adapting parameters, the processor 20compares the determined adapting parameters with the existing adaptingparameters at step 3130. If there is a significant change, the processor20 alerts the user by, for example, demanding a new identification,demanding an additional authentication method, activating a camera,sending an alert, making a remark in the system or a log file indicatingthat the dynamic signature has been altered by a suspicious user, or anycombination thereof.

The editing session ends at step 3120 and the virtual keyboard mayautomatically disappear.

In an exemplary embodiment, each virtual key is divided in to severalareas, such as 5 areas, center, top-right, top-left, bottom-right,bottom-left, as shown in FIG. 32 in creating the vectors, and theprocessor 20 monitors only a predefined set of keys, for example, 10predefined keys. The principles of the invention are not limited to 10keys, any number or all of the keys can be used, and any number ofsubdivisions can be used in a key. With this example, there are 5 to thepower of 10 variations and more than sufficient to distinguish differentusers. A significant change is that 20 percents of the monitored keyshave a change of the magnitude of a vector of more than 20 percents.

In one embodiment, the processor 20 can simply monitor the rhythm oftyping and records that rhythm as part of the dynamic signature of auser. The simple “thumbing,” the “hunt and peck” using, for example, twoor three fingers, typing by a novice user, and professional typingcertainly deliver different rhythm. A change of rhythm by 20 percents isconsidered significant. In yet another embodiment, both rhythm andvector changes are monitored.

Since process 3100 is preferred to be running in the background, unawareby the user, process 3100 provides additional security to the system.

As described above, the present invention provides, among other things,a method for enabling a user to use a virtual keyboard even if the somekeys in the virtual keyboard are partially or completely invisible, amethod for adapting a virtual keyboard to match the sizes and positionsof the fingers, a method for dynamically adapting each key to match thetyping habit of a user represented by a signature, and a method foradding keys to a virtual keyboard. Each of the added keys can also bedynamically adapted. While this invention has been described as having apreferred design, the present invention can be further modified withinthe spirit and scope of this disclosure. This application is thereforeintended to cover any variations, uses, or adaptations of the inventionusing its general principles. Further, this application is intended tocover such departures from the present disclosure as come within knownor customary practice in the art to which this invention pertains andwhich fall within the limits of the appended claims.

1. A touch screen device (1) comprising: a screen; touch-sensitive elements, a first portion of said touch-sensitive elements being arranged inside said screen and a second portion of said touch-sensitive elements is arranged outside of said screen; and a processor coupled to said screen and said touch-sensitive elements, said processor outputting a first virtual key to a first portion of said screen and moving said first virtual key from said first portion of said screen responsive to a user input to a position in which said first virtual key is partially displayed on said screen and covers a first portion of said first portion of said touch-sensitive elements and an invisible portion of said first virtual key covers a first portion of said second portion of said touch-sensitive elements, said first virtual key if hit causing said processor to perform a particular function.
 2. The touch screen device of claim 1, wherein said processor is operative to detect a signal indicating which one of said touch-sensitive elements has been hit.
 3. The touch screen device of claim 2, wherein responsive to detecting of said first portion of said first portion of said touch-sensitive elements being hit, said processor executes said particular function.
 4. The touch screen device of claim 2, wherein responsive to detecting of said first portion of said second portion of said touch-sensitive elements being hit, said processor executes said particular function.
 5. The touch screen device of claim 2, wherein said processor moves said first virtual key to a position in which all of said first virtual key is invisible and covers a second portion of said second portion of said touch-sensitive elements.
 6. The touch screen device of claim 5, wherein responsive to detecting of said second portion of said second portion of said touch-sensitive elements being hit, said processor executes said particular function.
 7. The touch screen device of claim 2, wherein said processor outputs a virtual keyboard including said first virtual key and moves said virtual keyboard to a position in which some virtual keys including said first virtual key are partially displayed on said screen and fall on said first portion of the touch-sensitive elements, and invisible portions of said some virtual keys fall on said second portion of said touch-sensitive elements.
 8. The touch screen device of claim 7, wherein responsive to detecting of said first portion of said first portion of said touch-sensitive elements being hit, said processor executes said particular function.
 9. The touch screen device of claim 7, wherein responsive to detecting of said first portion of said second portion of said touch-sensitive elements being hit, said processor executes said particular function.
 10. The touch screen device of claim 7, wherein said processor moves said virtual keyboard to a position in which said some virtual keys are completely invisible and fall on said second portion of said touch-sensitive elements.
 11. The touch screen device of claim 10, wherein said first virtual key covers a second portion of said second portion of said touch-sensitive elements, and responsive to detecting of said second portion of said second portion of said touch-sensitive elements being hit, said processor executes said particular function.
 12. The touch screen device of claim 7, wherein said virtual keyboard comprises a plurality of rows, and said some keys are arranged in one of said rows.
 13. The touch screen device of claim 1, wherein said first and second portions are next to each other in a consecutive manner.
 14. The touch screen device of claim 1, wherein said first and second portions are non-consecutive.
 15. The touch screen device of claim 14, wherein said screen is located in a first side of said touch screen device and said second portion is located on a second side other than said first side of said touch screen device.
 16. The touch screen device of claim 15, wherein said second portion includes a second key and said processor displays on said screen a third key indicating a location of said second key in said second side.
 17. The touch screen device of claim 16, wherein when one of said second and third keys is hit, said processor performs a second function assigned to said second key.
 18. A method for presenting a virtual keyboard comprising a plurality of keys on a touch screen device comprising a screen and touch-sensitive elements, a first portion of said touch-sensitive elements being arranged inside said screen and a second portion of said touch-sensitive elements is arranged outside of said screen, said method comprising: presenting said virtual keyboard, wherein all of said virtual keyboard is displayed on said screen and falls on said first portion of said touch-sensitive elements; moving said virtual keyboard, in response to a user input, to a position in which some keys including a first key of said virtual keyboard are partially displayed on said screen and are visible, said visible portions of said some keys fall on said first portion of said touch-sensitive elements and invisible portions of said some virtual keys fall on said second portion of said touch-sensitive elements, said first key covers a first portion of said first portion of said touch-sensitive elements and a first portion of said second portion of said touch-sensitive elements; and when said first key is hit, performing a particular function assigned to said first key.
 19. The method of claim 18, wherein said virtual keyboard comprises a plurality of rows, and said some keys are arranged in one of said rows.
 20. The method of claim 19, said performing step is performed if a hit on one of said first portion of said first portion of said touch-sensitive elements and said first portion of said second portion of said touch-sensitive elements is detected.
 21. The method of claim 19, said moving step moves said virtual keyboard to a position in which said some virtual keys are completely invisible and fall on said second portion of said touch-sensitive elements.
 22. The method of claim 21, wherein said first key covers a second portion of said second portion of said touch-sensitive elements and if a hit on said second portion of said second portion of said touch-sensitive elements is detected, said performing step is performed.
 23. The method of claim 18, wherein said screen is located on a first side of said touch screen device, said second portion of said touch-sensitive elements is located on a second side of said touch screen device other than said first side, said second portion of said touch-sensitive elements include a second key, and said method further comprises displaying on said screen a third key indicating a location of said second key in said second side.
 24. The method of claim 23, further comprising when said second key is hit, performing a second function assigned to said second key.
 25. The method of claim 24, further comprising when said third key is hit, performing a second function assigned to said second key. 